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Abstract 


This  research  addresses  the  implementation  of  an  electronic  element, 
which  emulates  the  biological  synaptic  interconnection,  in  an  artificial 
electronic  neural  system.  The  basic  interconnection,  or  the  weight,  consists 
of  an  electrically  reprogrammable,  nonvolatile,  analog  conductance  which 
programs  at  5V  levels.  In  addition,  the  fabrication  technology  for  this 
synaptic  interconnection  is  compatible  with  existing  CMOS  VLSI  processes. 
The  attractive  features  of  this  synaptic  weight  will  be  discussed  in  this 
report.  Furthermore,  this  report  examines  the  material  needs,  the  device 
structures,  the  use  of  the  synaptic  weights  in  a  two-tap  weight  linear 
adaptive  neural-like  circuit  and  the  issue  of  integrating  both  the  synaptic 
weight  elements  and  the  peripheral  circuit  onto  a  single  silicon  wafer. 


1.  Introduction 

The  current  surge  of  enthusiasm  for  neural  network  aims  to  construct  systems  that  can  learn 
or  modify  their  behavior  according  to  the  environment.  There  are  many  similarities  which  exist 
between  this  new  class  of  machine  and  human  beings.  One  of  these  similarities  is  the  massive 
parallelism  in  processing  information.  Parallel  processing1  concepts  are  in  stark  contrast  to  the 
operations  of  modem  digital  computers  that  perform  large  numbers  of  sequential  operations  very 
rapidly  and  accurately. 

Researchers  believe  the  synaptic  junctions  in  a  neural  system  are  the  local  memory  sites 
and  provide  the  physiological  basis  for  the  distributed  parallel  systems.2, 3  These  synapses  are  not 
only  modifiable  but  also  serve  the  functions  of  storing  and  transmitting  information  from  neuron  to 
neuron.  To  reduce  the  complex  modelling  required  for  the  synaptic  interconnection,  the 
representation  of  the  synapse  has  been  simplified  to  a  single  ideal  junction  between  the  output  of 
neurons  (axons)  and  the  inputs  to  neurons  (dendrites).  Synaptic  modification  requires  information 
from  the  input  and  the  output  of  the  neuron  in  order  to  perform  complex  recognition.  Therefore,  the 
nature  of  the  synaptic  junction  and  the  principle  or  algorithm  which  controls  local  organization  at 
the  neuron  level  become  two  central  issues  pertaining  to  neural  networks  research. 

The  recent  interest  in  neural  networks4, 5  is  a  direct  consequence  of  the  programmability 
which  is  an  essential  feature  of  learning  machines,  associative  memories,  and  adaptive  signal 
processors.  Programmability  requires  a  modification  of  the  synaptic  strength  in  the  language  of 
neurobiology.  If  we  seek  an  efficient  hardware  implementation  of  electronic  neural  systems,  then 
the  synapses  -  as  well  as  the  network  itself-  should  be  analog.  Several  attempts  have  been  made  to 
realize  programmable  synapses,  either  digitally6  or  with  temporary  storage  on  the  input  capacitance 
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of  a  MOS  Transistor7, 8  to  alter  the  latter’s  analog  conductance.  The  former  approach  stores  the 
weight  information  in  digital  registers  and  thus  suffers  from  excessive  chip  area  and  power 
consumption.  On  the  other  hand,  although  the  MOS  Transistor  provides  an  analog  synaptic 
strength  (weight)  in  a  small  chip  area,  the  weight  is  temporary  and  requires  periodic  refresh  similar 
to  a  DRAM.  Thus,  this  dynamic  refresh  approach  lacks  the  nonvolatility  and  storage  properties  of 
an  EEPROM  cell.  Researchers  at  Intel  have  reported  an  electrically  trainable  artificial  neural 
network  with  floating  gate  device  as  the  synaptic  element.9  Although  the  floating  gate  device  has 
the  property  of  nonvolatility,  its  high  programming  voltage  requirement  prevents  it  from  being 
technologically  compatible  with  scaled  CMOS  process. 

In  this  research  report  we  describe  a  new  approach  to  obtain  an  electrically  reprogrammable  or 
modifiable  synaptic  weight  to  be  used  as  a  basic  functional  element  in  electronic  neural  systems. 
The  salient  features  of  this  network  element  are  the  following: 

•  Low  programming  voltages(5-10V)  which  are  compatible  with  peripheral 
CMOS  VLSI  technology  in  contrast  with  Floating  Gate  approaches. 

•  Low  power  dissipation  ( <  1  pW). 

•  Dynamic  Range  of  1000:1  (60  dB). 

•  Nonvolatile  features  which  mimic  biological  synapses  with  respect  to  memory 
loss  (e.g.  20%  of  the  information  available  after  10  years)  and  reinforced 
learning  (e.g.  successive  interrogation  enhances  memory  retention). 

•  Small  synaptic  area  on  a  VLSI  chip  (e.g.  less  then  20pm2  for  1-25  pm  feature 
sizes). 

•  Extensive  erase/write  programming  cycles  are  possible  with  this  synapse  ( > 

108  cycles)  in  contrast  with  Floating  Gate  approaches. 

•  Inherent  radiation  damage  resistance  beyond  a  total  dosage  of  IMRad  (Co60) 
and  10°  Rad/sec  transient  which  is  not  possible  with  Floating  Gate  technology. 

Thus,  if  radiation  damage  resistance  of  neural  networks  is  an  important  issue, 
then  the  SONOS  devices  have  demonstrated  success  in  this  area. 

The  basic  nonvolatile  device  structure,  which  we  describe  in  this  report  was  first  introduced  as 
a  digital  nonvolatile  memory  cell  in  the  summer  of  1987  at  the  IEEE  Device  Research  Conference10 
by  researchers  at  Lehigh  University.  We  have  had  a  continual  involvement  over  a  20  year  period 
with  nonvolatile  memories,  beginning  in  the  late  60’s  where  we  had  programming  voltages  of  25V,  to 
the  late  80’s  with  our  novel  5V  SONOS  device  structures.  During  this  time  period  we  introduced  the 
use  of  CCD’s  and  nonvolatile  memories11, 12, 13  in  nonvolatile  charge  addressed  memories 
(NOVCAM).  These  ideas  have  been  employed  recently  for  neural  network  circuits  by  researchers  at 
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Lincoln  Laboratories.14  Our  recent  work  recognizes  the  inherent  analog  conductance  aspect  of  the 
nonvolatile  SONOS  memory  device  which  makes  it  a  perfect  candidate  for  the  modifiable  synapse  in 
an  electronic  neural  system. 

In  addition  to  the  realization  of  an  electronic  element  to  simulate  the  synaptic  interconnections 
of  a  neural  network,  we  must  have  a  method  or  algorithm  to  change  or  reprogram  these 
interconnections  and,  thus,  alter  the  connectivity  of  the  neural  network.  We  have  had  experience 
with  a  particular  form  of  an  algorithm,  namely,  the  Widrow-Hoff  Least  Mean  Square  (LMS)15  error 
algorithm  or  in  neural  network  terminology  -  the  so-called  ’delta  rule’.  In  the  late  70’s  we  researched 
a  CCD  Adaptive  Analog  Signal  Processor16, 17  which  realizes  the  ’delta  rule’  with  CCD  analog  delay 
lines  and  electrically  reprogrammable  MNOS  analog  conductance  weights.  These  weights  were 
nonvolatile  memory  transistors  whose  analog  conductance  was  programmed  with  voltages  ranging 
from  15-25V.  Our  recent  work  on  ’scaling’  these  programmable  analog  conductances  has  resulted  in 
a  new  device  structure,  called  the  SONOS  nonvolatile  memory  transistor,  which  can  be 
reprogrammed  with  voltages  ranging  from  5-10V.  This  work  has  recently  been  described  at  the  1991 
11th  IEEE  Nonvolatile  Semiconductor  Memory  Workshop.18  These  voltage  levels  are  compatible 
with  ’scaled’  CMOS  VLSI  technology  which  has  12- 15V  breakdown  voltages  for  1.25pm  feature  sizes. 
In  this  report  we  describe  our  recent  work  on  the  electrically  reprogrammable  (modifiable)  SONOS 
nonvolatile  synapse  and  a  simple  electronic  neuron  with  2  synaptic  weights.  We  discuss  this  two-tap 
weight  linear  adaptive  neuron  in  terms  of  the  technology,  the  electrical  characteristics  of  the 
synapses,  and  their  performance  in  this  simple  test  vehicle  -  a  'delta  rule’  adaptive  signal  processor. 

2.  Technology  and  Characterization  of  the  SONOS  Synaptic  Weight 

The  programmable  synapse  is  the  result  of  an  ongoing  effort  at  Lehigh  University  to  ’scale'  the 
programming  voltages  required  to  alter  the  analog  conductance  of  a  nonvolatile  memory  transistor 
with  a  multi-layer  (oxide-nitride-oxide)  gate  insulator  as  shown  in  Fig.  1.  Recent  efforts  in  scaling 
this  device  have  resulted  in  a  SONOS  (Silicon/Blocking  Oxide/Nitride/Tunneling  Oxide/Silicon) 
nonvolatile  memory  transistor  which  is  electrically  reprogrammable  at  CMOS  voltage  levels. 
Typically,  the  tunneling  oxide  is  15-25&  ,  the  storage  nitride  is  50-100X  and  the  blocking  oxide  is 
35-50X  .  Fig.2  shows  the  Transmission  Electron  Microscope  (TEM)  photograph  of  the  cross  sectional 
view  of  the  SONOS  transistor.  This  device  is  similar  to  a  SNOS  transistor  except  for  the  addition  of 
the  blocking  oxide  which  is  used  to  inhibit  injection  of  carriers  from  the  polysilicon  gate  electrode 


3 


and  also  to  improve  the  memory  retention  by  prohibiting  the  transfer  of  stored  charge  from  the 
nitride  to  the  gate  electrode.  As  a  result,  the  blocking  oxide  permits  the  entire  dielectric  sandwich  to 
be  scaled  to  dimensions  where  programming  voltages  ranging  from  5-10  V  are  possible. 


When  the  SON  OS  device  is  subjected  to  a  positive  (or  negative)  programming  pulse,  electrons 
(or  holes)  are  injected  into  the  silicon  nitride  layer  by  means  of  tunneling  across  the  thin  tunnel 
oxide.  The  injected  charges  are  trapped  by  the  silicon  nitride  and,  thus,  shift  the  threshold  voltage 
positively  (or  negatively).  The  threshold  voltage  of  a  SONOS  transistor  can  be  written  as 


VTH  ~  <f,GS~fP"+(  ~  + 
'-'eff 
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-X 
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)  Qn  +  24>b 


esi  q  NB 


■'eff 


0) 


where  <{>B  is  the  bulk  potential,  <J>GS  is  the  gate  to  semiconductor  workfunction,  Qf  is  the  fixed  charge 
at  the  tunneling  oxide-silicon  interface,  eox  and  eN  are  the  dielectric  permittivities  of  the  oxide  and 
nitride,  eai  is  the  dielectric  permittivity  of  the  bulk  silicon,  xot  is  the  tunnel  oxide  thickness,  xob  is  the 
blocking  oxide  thickness,  Xjj  is  the  nitride  thickness,  Xis  the  charge  centroid  in  the  insulator,  and  Q^- 
is  the  charge  stored  in  the  nitride,  NB  is  the  bulk  doping  density,  and 


Ceff  = 


"ox 
— I 

eN 


xot  +  — ^  +  Xob 


(2) 


We  assume  the  tunnel  oxide  and  blocking  oxide  have  the  same  dielectric  permittivity,  even  though,  it 
is  known  that  the  tunnel  oxide  is  silicon  rich  and  the  blocking  oxide  is  an  oxynitride.  The  values  of 
the  charge  centroid  X  and  the  variable  charge  stored  in  the  nitride  QN  will  change  as  the  device  is 
written  or  erased.  The  analog  conductance  of  the  SONOS  synaptic  weight  is  given  as 

8ds  =  Peff  1  Ceff  (VGS  "  VTH)  (3) 


where  peff  is  the  effective  carrier  mobility,  is  the  read  voltage,  and  is  the  electrically 
modifiable  threshold  voltage  given  in  equation  (1).  Therefore,  there  are  two  ways  which  the  analog 
channel  conductance  can  be  altered:  (1)  change  the  value  of  Vqq  or  (2)  change  the  value  of  Vjjj  by 
altering  the  stored  charge,  QN,  in  the  nitride.  In  our  study,  the  latter  approach  is  chosen. 

The  SONOS  transistors  have  been  characterized  for  their  memory  properties  with  the  test 
station  described  by  Roy  et.  al. 19  This  test  station  allows  one  to  take  both  erase/write  and  retention 
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measurements.  To  investigate  the  memory  loss/retention  properties  of  the  synaptic  weight  element, 
retention  measurements  are  taken.  The  retention  characteristics  are  obtained  by  applying  positive 
(negative)  five  volts  to  the  gate  for  10  seconds  to  place  the  device  in  the  write  (erase)  state  and  then 
measuring  the  tum-on  voltage  after  a  varying  delay  time.  The  tum-on  voltage  is  related  to  the 
threshold  voltage  by 

Vt  =  Vth  +  ^  (4) 

with  IDS  as  the  forced  drain  to  source  current  during  measurement  and 

Mii,ff(£)c«ff  (5) 

where  W  is  the  width  of  the  transistor,  £  is  the  length  of  the  transistor,  and  peff  is  the  effective 
mobility.  The  effective  mobility  is  the  bulk  mobility  reduced  by  Coulombic  and  surface  scattering  of 
carriers  in  the  inversion  layer.  This  mobility  is  influenced  by  the  gate  and  substrate  voltages.20  For 
a  SONOS  transistor,  retention  measurements  indicate  that  greater  than  20  percent  of  the  memory 
window  remains  after  a  projected  10  year  delay  time  as  shown  in  Fig.  3.  The  erase/write 
measurements  indicate  the  programming  speed  of  the  synaptic  weight  element.  To  measure  the 
writing  (erasing)  speed,  negative  (positive)  five  volts  are  applied  to  the  gate  for  10  seconds  to  place 
the  device  in  the  erase  (write)  state.  Then,  positive  (negative)  five  volts  are  applied  to  the  gate  with 
varying  pulse  widths  and  the  tum-on  voltage  is  measured  after  each  pulse  width.  The  erase/write 
characteristics  of  the  SONOS  memory  transistor  are  shown  in  Fig.  4.  A  wide  dynamic  range  is  one 
of  the  essential  properties  for  the  synaptic  weight  element,  and  Fig.  5  illustrates  a  60  dB  in  dynamic 
range  after  ±5V  programming  for  the  SONOS  synaptic  weight.  In  addition,  a  recent  study  in 
reliability  has  demonstrated  the  inherent  resistance  of  the  SONOS  memory  transistor  to  radiation 
damage  (SV-jh  =  0.1V,  with  Vqq  =  +  5V  at  lMRad  Co60  radiation).21 

3.  Single-level  Linear  Adaptive  Neuron 

We  have  incorporated  the  SONOS  synaptic  weights  into  a  single-level  linear  neuron-like 
circuit  using  a  Widrow-Hoffs  delta  learning  rule.15  The  circuit  is  built  with  a  hybrid  breadboard  of 
CMOS  components  for  the  control  logic  and  the  algorithm  implementation  and  the  SONOS 
nonvolatile  memory  transistors  to  demonstrate  the  voltage  level  compatibility  of  both  SONOS  and 
CMOS  technologies.  Many  researchers  believe  that  the  neural  system  is  made  up  of  several  layers’ 
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of  neurons  and  Fig.  6  shows  the  multi-layer  architecture  of  an  artificial  neural  network.  The  first 
layer  of  neurons,  the  input  layer,  can  be  best  thought  as  the  sensory  neurons  in  a  human  body.  The 
weight  connections  between  the  input  layer  and  the  middle  hidden  layer  are  normally  considered  to 
be  feedforward  and  fixed.  On  the  other  hand,  the  weight  connections  between  the  middle  hidden 
layer  and  the  output  layer  are  considered  to  be  feedback  in  nature.  Our  work  has  concentrated  on 
the  implementation  of  two  neurons  in  the  hidden  layer  and  one  output  neuron  as  highlighted  in  the 
figure. 

Fig.  7  shows  the  block  diagram  of  the  single-level  linear  adaptive  neuron.  A  desired  response 
(or  external  teacher),  d(m),  is  presented  to  the  neuron  as  the  training  signal.  If  the  output  of  the 
linear  adaptive  neuron  is  not  trained,  then  there  exists  a  mismatch  between  the  output  of  the  linear 
adaptive  neuron,  y(m),  and  the  desired  response,  dim), 

£(m)  =  d(m)  -  y(m)  (6) 

where  £(m)  is  the  error  generated.  This  error  is  then  used  by  a  learning  algorithm,  namely  the 
Clipped-data  Least  Mean  Square  Error  algorithm,  to  minimize  the  error  generated  and  thereby 
training  the  neuron  to  the  correct  response.  This  single-level  linear  adaptive  neuron  has  two  tap 
weights,  each  weight  composed  of  two  SONOS  analog  electrically  reprogrammable  conductances  as 
shown  in  Fig.  8.  Since  the  synaptic  weight  may  be  either  positive  or  negative  in  value,  we  have 
chosen  a  differential  weighting  scheme.  If  the  analog  conductance  connecting  the  positive  summing 
path  to  the  differential  operational  amplifier  is  greater  than  the  analog  conductance  connecting  the 
negative  summing  path  to  the  differential  operational  amplifier,  then  the  weight  is  positive  in  value. 
On  the  other  hand,  if  the  opposite  case  is  true,  then  the  weight  is  negative  in  value.  A  positive 
weight  value  corresponds  to  an  excitatory  synaptic  strength  and  a  negative  weight  value  corresponds 
to  am  inhibitory  synaptic  strength. 

In  operation,  the  input  signal  x(t)  is  passed  through  a  switched  capacitor  analog  delay  line 
where  the  input  signal  is  sampled  and  delayed  to  create  four  tapped  signal  outputs  x0(m),  .tj (m),  x^m), 
and  .tjCm).  These  tapped  signals  multiply  to  their  corresponding  programmable  weights  VV0,  Wj,  W2, 
and  Wy  and  the  result  is  summed  linearly  at  the  summing  amplifier.  The  output  y(m)  can  be 
expressed  as 
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y(m)  =  £  Wk(m)  xm_k 
i= 0 

where  m  is  the  time  index  and  k  is  the  spatial  index.  A  correlated  double  sampling  technique22  is 
employed  in  the  circuit  to  remove  the  unwanted  noise  and  offset  voltages  introduced  by  the 
operational  amplifiers  and  switching  circuits.  The  linear  adaptive  neuron  is  configured  to  perform  a 
Widrow-Hoffs  delta  rule  as 

Wk(m+ 1 )  =  Wk(m)  +  A  Wk(m)  (8) 

where  A  W(m)  is  the  incremental  weight  to  be  calculated  by  the  clipped-data  least  mean  square  error 
(C-LMSE)  algorithm23: 

A  Wk(m)  =  2(1  |e(m)|  Sgn[£(m)]  Sgn[x(m-k)]  (9) 

where  p  is  the  convergence  factor.  Compared  to  the  regular  Least  Mean  Square  Error  algorithm,  the 
input  signal  amplitude  is  clipped  in  the  learning  algorithm.  This  algorithm  eliminates  the  usage  of  a 
four  quadrant  multiplier  needed  for  the  LMS  error  algorithm.  The  sign  multiplication  in  the 
incremental  weight  calculation  is  essentially  an  Exclusive  OR  operation  and  the  output  of  the 
Exclusive  OR  gate  controls  the  path  of  proper  gate  programming  voltage  for  the  SONOS  synaptic 
weight.  If  the  convergence  factor  is  small,  then  the  system  will  minimize  the  misadjustment  caused 
by  the  variance  of  the  weights;  however,  this  also  results  in  a  long  convergence  time.  Conversely,  if 
we  choose  to  use  a  larger  convergence  factor,  then  the  convergence  time  of  the  system  is  shortened 
with  the  penalty  of  larger  misadjustment.  The  backpropagating  error  is  used  to  calculate  the 
adjustments  to  minimize  the  system  error  as  shown  in  equation  (9).  Once  the  error  is  minimized, 
the  system  is  said  to  be  in  its  steady  state  condition24  where  the  output  of  the  system,  y(m),  is  the 
best  match  of  the  training  signal,  d(m),  or  the  ’external  teacher". 

The  incremental  weight  update  is  essentially  a  cross  correlation  between  the  error  and  the 
clipped  input  data  vectors.  The  update  stops  when  the  two  vectors  become  orthogonal.  Sometimes, 
the  network  may  be  overcorrected  initially,  however,  the  error  will  be  quickly  minimized  by- the 
learning  algorithm  and  the  system  reaches  its  desired  response.  The  digital  delay  line  provides  the 
sign  information  of  the  input  to  the  learning  algorithm.  A  special  steering  network  is  designed  to 
switch  the  proper  programming  voltages  to  the  gate  terminals  of  the  SONOS  transistors  once  the 
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incremental  weights  are  calculated. 


4.  Experimental  Results 

There  are  two  main  types  of  characteristics  from  which  the  electrical  performance  of  the  linear 
adaptive  neuron  can  be  evaluated.  The  first  characteristic,  namely  the  output  and  training  signals 
versus  time  characteristics,  gives  the  information  on  how  well  the  output  signal  approximates  the 
training  signal  especially  in  the  phase  relationship  between  these  two  signals.  The  second 
characteristic,  namely  the  error  signal  versus  time  characteristics,  shows  how  fast  the  linear 
adaptive  neuron  adapts  before  it  reaches  its  minimum  error.  A  typical  output  and  training  signals 
versus  time  characteristic  consists  of  two  parts:  the  initialized  and  the  adapted  part.  In  the 
initialized  part,  the  weights  are  first  initialized  to  a  known  state  (either  the  fully  positive  or  the  fully 
negative  state)  and  then  the  weights  are  subjected  to  a  reading  voltage  to  read  out  the  weight 
information  and  the  output  signal  and  the  training  signal  are  compared  and  recorded.  The  linear 
adaptive  neuron  is  then  allowed  to  adapt  itself  to  the  training  signal  and  the  results  are  shown  in 
the  adapted  part  of  the  characteristics.  Figure  9  shows  the  output  and  training  signal  versus  time 
characteristic. 

A  typical  error  signal  versus  time  characteristic  is  obtained  with  initialized  weight  values  and 
monitoring  the  error  signal  with  time.  Our  observation  indicates  the  weight  initialization  scheme 
affects  the  convergence  behavior  of  the  linear  adaptive  neuron.  This  phenomenon  is  attributed  to 
the  nonsymmetric  erase  and  write  characteristics  of  the  SONOS  transistor.  Therefore,  one  weight 
initialization  scheme  may  require  more  erase  action  taking  place  than  another  weight  initialization 
scheme,  causing  a  difference  in  convergence  characteristics.  Figures  10  shows  a  typical  error  versus 
time  characteristic. 

5.  Technical  Achievements 

During  the  period  of  investigation,  several  technical  achievements  have  been  accomplished. 
Since  the  programming  characteristics  of  the  SONOS  synaptic  weight  elements  strongly  govern  the 
performance  of  the  integrated  solid-state  linear  adaptive  neuron,  the  optimization  of  the  SONOS 
synaptic  weight  element  becomes  one  of  the  key  issues  of  this  research  effort.  We  have  started  our 
research  with  a  SONOS  device  structure  of  20X  of  tunneling  oxide,  96^  of  nitride,  and  25X  of 
blocking  oxide.  The  cross-over  time  for  this  structure  is  1  second.  After  examining  the  programming 
behavior  of  the  SONOS  structure  mentioned  above,  we  have  decided  to  scale  down  the  nitride 
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thickness  and  increase  the  blocking  oxide  thickness.  This  scaling  scheme  is  based  on  the  analysis 
which  promises  higher  programming  field  across  the  multi-layer  dielectrics  and  better  charge 
retention  in  the  nitride  due  to  the  elimination  of  the  carrier  injection  from  the  gate  terminal  as  well 
as  the  carrier  tunneling  to  the  gate  terminal.  The  scaling  effort  has  produced  a  new  device  structure 
with  the  programming  speed  improvement  of  one  order  of  magnitude  compared  to  the  previous 
version.  We  have  then  incorporated  these  new  SONOS  synaptic  weight  elements  in  the  linear 
adaptive  neuron  and  observed  a  corresponding  one  order  of  magnitude  improvement  in  convergence 
speed.  Therefore,  the  direct  relationship  between  the  programming  characteristics  of  the  SONOS 
synaptic  weight  elements  and  the  performance  of  the  linear  adaptive  neuron  has  been  proven 
experimentally. 

Encouraged  by  the  sucess  in  scaling  down  the  device  dielectric  structure,  we  have  extended  our 
research  effort  in  fabricating  two  new  sets  of  devices.  One  set  of  devices  have  the  dielectric 
thicknesses  similar  to  Nozaki  et.  al .25  with  18\  tunneling  oxide,  49\  nitride  and  40X  of  blocking 
oxide.  The  other  set  of  the  devices  have  an  ultra-thin  tunneling  oxide  of  llX  ,  49X  of  nitride  and 
40X  of  blocking  oxide.  For  the  first  time,  the  ultra-thin  tunneling  oxide  SONOS  devices  have  been 
successiully  tested  and  reported.  The  programming  characteristics  of  the  ultra-thin  tunneling  oxide 
indicate  a  much  better  improvement  over  those  published  in  the  literature  and  the  result  is  shown  in 
figure  10.  In  addition,  a  novel  device  structure  is  currently  under  investigation,  namely  the  buried 
channel  SONOS  device  structure.  This  structure  has  demonstrated  better  programming  speed  as 
well  as  unproved  retention  time  compared  to  the  conventional  surface  channel  SONOS  device  with 
similar  dielectric  dimensions.  A  typical  buried  channel  SONOS  device  programming  characteristic 
is  shown  in  figure  11. 

Furthermore,  a  theoretical  analysis  of  the  convergence  behavior  with  a  variable  convergence 
factor  has  been  developed.  The  variable  convergence  factor  scheme  is  a  direct  result  of  using  the 
SONOS  memory  transistors  as  the  reprogrammable  synaptic  weight  elements.  The  convergence 
factor  initially  starts  with  a  large  value,  which  accelerates  the  convergence  process.  As  time 
progress,  the  convergence  factor  reduces  its  value  and  aids  the  linear  adaptive  neuron  converging  to 
its  optimum  steady  state  condition.  The  analysis  is  composed  of  two  separate  models: 
ERASE/WRITE  tunneling  model  and  Fowler-Nordheim  tunneling  model.  A  computer  software  has 
been  written  to  simulate  the  convergence  behavior  of  the  linear  adaptive  neuron  with  the 
incorporation  of  variable  convergence  factor.  Since  the  device  model  is  physically  based,  the  input 
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variables  of  the  simulation  software  are  actual  physical  device  parameters  of  the  SONOS  synaptic 
weight  elements. 

A  fully  computer  controlled  data  acquisition  system  is  an  invaluable  tool  for  SONOS  synaptic 
weight  element  characterization.  Previously,  the  measurement  system  required  the  operator  to 
manually  set  up  the  measurement  sequence  and  hand-recorded  the  data  obtained.  An  automated 
data  acquisition  system  enables  the  user  to  set  up  measurements,  analyze  the  data,  and  extract 
device  parameters,  all  under  the  control  of  one  console.  The  automated  data  acquisition  system  has 
been  designed,  constructed  and  fully  tested.  A  block  diagram  of  the  system  is  depicted  in  figure  12 
and  the  schematic  of  the  HPIB  command/data  interpreter  is  shown  in  figure  13.  The  system  is 
composed  of  an  HP  9836  technical  computer  served  as  the  controller,  a  HPIB  (HP  Interfi.  e  Bus) 
command/data  interpreter  which  interfaces  with  the  computer  and  sets  up  tne  erase/write/read 
circuit  and  the  pattern  generator,  a  digital  storage  oscilloscope  responsible  for  caphiring  the 
measured  result  and  transmitting  the  data  back  to  the  computer  for  analysis.  In  addition,  the  wafer 
can  be  placed  in  an  automatic  wafer  prober  with  temperature  controller  to  perform  wafer  level 
temperature  testing.  A  software  control  routine  has  been  written  to  coordinate  the  instruments  in 
the  system.  The  source  code  for  control  routine  can  be  provided  upon  request. 

Integration  of  the  linear  adaptive  neuron  onto  a  single  silicon  wafer  is  one  of  the  mail  goals  of 
our  research  efforts.  We  have  acquired  a  computer  aided  design  software  package,  developed  by  the 
Mentor  Graphics  Corporation,  and  implemented  on  our  SUN  Sparc  Workstations.  The  first  task  of 
using  this  software  is  to  develop  a  technology  file  geared  to  the  fabrication  sequence  of  the 
Microelectronic  Research  Laboratory  at  Lehigh.  In  addition,  the  technology  file  must  accommodate 
both  the  conventional  CMOS  process  and  the  Nonvolatile  Semiconductor  Memory  (NVSM) 
technology  for  the  SONOS  synaptic  weight  element  implementation.  Novel  processing  steps  such  as 
buried  channel  implants,  semiconductor  implanted  resistor  are  also  incorporated  into  the  technology 
file.  Since  we  are  creating  analog  ASICs,  area  and  power  consumption  must  be  minimized.  We  have 
adopted  a  hierarchical  design  approach  from  basic  functional  cell  design  up  to  the  entire  integrated 
solid-state  linear  adaptive  neuron  design  to  ensure  the  minimization  of  power  and  area  consumption. 
The  design  of  the  entire  integrated  adaptive  neuron  has  been  completed  and  the  process  of  making 
the  masks  containing  the  design  is  currently  undergoing. 

The  integrated  adaptive  neuron  is  composed  of  five  main  cells,  namely,  the  clock  module,  the 
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analog  delay  line  module,  the  steering  network  module,  the  summing  module  and  the  algorithm 
module.  The  clock  module  generates  all  the  controlling  signals  and  thus  synchronizes  all  the 
operations  of  the  linear  adaptive  neuron  to  a  master  clock.  The  analog  delay  line  module  utilizes 
switched  capacitor  scheme  to  delay  the  input  signal.  The  steering  network  module  is  responsible  to 
direct  proper  programming  voltages  to  the  SONOS  synaptic  weight  elements  during  adaptation. 
The  summing  module  sums  up  the  weighted  input  signals  and  removes  the  unwanted  noise  from  the 
system.  The  algorithm  module  uses  the  information  from  the  summing  module  to  produce 
programming  voltages  for  the  steering  network  module  according  to  the  clipped  data  Least  Mean 
Square  error  algorithm.  A  complete  layout  design  is  shown  in  figure  14.  A  printed  circuit  board 
version  of  the  integrated  solid-state  linear  adaptive  neuron  is  also  designed  and  implemented.  The 
schematic  of  the  PCB  version  of  the  linear  adaptive  neuron  can  be  furnished  upon  request. 

We  believe  we  have  advanced  the  understanding  of  the  how  the  SONOS  synaptic  weight 
elements  can  be  used  in  the  implementation  of  the  neural  network.  In  addition,  we  have 
demonstrated  success  in  scaling  of  the  SONOS  nonvolatile  memory  transistors  and  thus  provided  a 
guideline  for  future  scaling  of  the  SONOS  devices.  We  have  also  contributed  to  the  state  of  the  art  in 
the  implementation  of  the  artificial  neural  networks  with  our  design  of  integrated  solid-state  linear 
adaptive  neuron.  Under  the  support  of  this  project,  numerous  papers  have  been  accepted  and  a  list 
of  publications  is  attached  in  appendix  B. 

6.  Conclusions 

The  SONOS  nonvolatile  memory  transistor  has  been  shown  to  be  an  ideal  electronic  element 
for  the  electrically  reprogrammable  analog  conductance  in  an  artificial  neural  network.  We  have 
demonstrated  the  attractive  features  of  this  synaptic  weight  for  the  use  of  large  neural  network 
systems,  for  instance,  low  programming  voltage  (5-10V),  low  power  dissipation(<lpW  /  synapse), 
small  chip  area  (estimated  20|im2/  weight  cell  for  a  1.2  |im  feature  size),  a  dynamic  range  of  60  dB, 
good  memory  retention  (20  %  window  at  a  projected  10  years  period),  and  endurance  beyond  10' 
erase/write  cycles.  In  addition,  the  SONOS  synaptic  weight  has  inherent  resistance  to  radiation 
damage  (AV,f/l  =  0.1V')  with  V^=+5V  at  IMRad  Co60  radiation).  We  have  been  continuing  our  efforts  in 
optimizing  the  modifiable  synaptic  weights  to  provide  better  electrical  characteristics  for  neural 
network  applications. 

We  have  also  incorporated  the  SONOS  synaptic  weights  into  a  single-level  two  tap  linear 
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adaptive  neuron  employing  a  Widrow-Hoffs  delta  learning  rule.  The  combination  of  CMOS  control 
circuits  and  SONOS  synaptic  weights  has  demonstrated  the  feasibility  of  integrating  these  two 
technologies  onto  a  single  silicon  wafer.  The  initial  results  are  encouraging  and  promising  and 
provide  insight  and  direction  into  the  integration  of  these  two  technologies  to  realize  large  artificial 
neural  network  systems. 
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Figure  1.  Cross  Sectional  View  of  the  (a)  MNOS  (b)  SONOS  Electrically 
Modifiable  Synaptic  Weight  (c)  SONOS  Ideal  Energy-Band  Diagram 
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Figure  2.  TEM  Photomicrograph  of  the  SONOS  Synaptic  Weight 
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Figure  3.  Retention  Characteristics  of  a  Modifiable  SONOS  Synaptic  Weight 
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Figure  4.  Erase/Write  Characteristics  of  a  Modifiable  SONOS  Synaptic  Weight 
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Figure  5.  Dynamic  Range  of  a  SONOS  Synaptic  Weight  After  Programming 
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Figure  6.  Conceptual  View  of  Multi-layer  Artificial  Neural  Network 
Architecture  Incorporating  the  Single-Level  Linear  Adaptive  Neuron 
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Figure  7.  Block  Diagram  of  a  Single-level  Linear  Neuron 
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Figure  8.  Electrical  Implementation  of  the  Synaptic  Weights 
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Figure  9.  Output  and  Training  Signals  versus  Time  Characteristics  of  a 
Two  Tap  Weight  Linear  Adaptive  Neuron  (a)  Initialized  (b)  Adapted 
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Figure  10.  Error  Signal  versus  Time  Characteristics  of  a  Two  Tap  Weight 

Linear  Adaptive  Neuron 


Figure  ll.Programming  Speed  of  the  Newly  Made  Synaptic  Weight  Element 
with  11  Angstrom  of  Tunneling  Oxide 
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Figure  12.  Programming  Speed  of  the  Novel  Buried  Channel  SONOS  Synaptic 
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Figure  13.  Block  Diagram  of  the  Automated  Data  Acquisition  System 
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Figure  15.  Complete  Layout  of  the  Integrated  Solid-State  Electronic 
Linear  Adaptive  Neuron 
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Appendix  A 

CMOS/NVSM  Fabrication  Sequence  with  Novel  Buried  Channel  Devices 

•  Starting  Material  3  in  p-type  2-3  Ohm/cm 

•  N-Well  Implant  Formation 

1.  RCA  Clean 

2.  Wet  Oxidation  for  1000  °A  ,  950  °C,  25  min 

3.  Photoresist  Application 

4.  Prebake,  98  °C,  30  min 

5.  Mask  Level  NW 

6.  Photoresist  Development 

7.  Postbake,  120  °C,  30  min 

8.  BHF  Etch,  10:1,  2  min 

9.  Implant,  Phosphorus,  4.8  xlO12,  lOOKeV 

10.  Plasma  Photoresist  Strip  (Oxygen) 

11.  Photoresist  Stripper 

12.  Dry  Oxidation,  500  ,  1200  °C,  5  min 

13.  Implant  Anneal,  1200  °C,  240  min 

•  Active  Device 

1.  RCA  Clean 

2.  LPCVD  Nitride,  200mTorr,  10:1  ratio,  1000  X  ,  54  min 

3.  Photoresist  Application 

4.  Prebake,  98  °C,  30  min 

5.  Mask  Level  AD 

6.  Photoresist  Development 

7.  Postbake,  120  °C,  30  min 

8.  Plasma  Etch  Nitride  (CF4) 

9.  Photoresist  Stripper 

•  Channel  Stop  Implant 

1.  Photoresist  Application 

2.  Prebake,  98  °C,  30  min 

3.  Mask  Level  FI 

4.  Photoresist  Development 
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5.  Postbake,  120  °C,  30  min 

6.  Implant,  BF2,  5X1011, 145KeV 

7.  Plasma  Photoresist  Strip  (Oxygen) 

8.  Photoresist  Stripper 

9.  RCA  Clean 

10.  Wet  Field  Oxidation,  6500  X  ,  1100  °C,  60  min 

11.  BHF  Etch,  10:1, 1  min 

12.  Hot  H3P04, 170°C,  35  min 

13.  BHF  Etch,  10:1,  1.5  min 

14.  RCA  Clean 

15.  Wet  Oxidation,  900  °C,  20  min 

16.  BHF  Etch,  10:1, 1  min 

17.  RCA  Clean 

18.  Wet  Pad  Oxidation,  900  °C,  15  min 

19.  Implant,  Boron,  9x  1011,  70KeV 

20.  BHF  Etch,  10:1,  2min 

21.  RCA  Clean 

22.  Anneal,  950  °C,  30  min 

•  Buried  Channel  Formation 

1.  RCA  Clean 

2.  Wet  Pad  Oxidation,  900  °C,  15  min 

3.  Photoresist  Application 

4.  Prebake,  98  °C,  30  min 

5.  Mask  Level  BCN 

6.  Photoresist  Development 

7.  Postbake,  120  3C,  30  min 

8.  Implant,  Arsenic,  5x  1011,  75KeV 

9.  Plasma  Photoresist  Strip  (Oxygen) 

10.  Photoresist  Stripper 

11.  Photoresist  Application 

12.  Prebake,  98  °C,  30  min 

13.  Mask  Level  BCP 

14.  Photoresist  Development 

15.  Postbake,  120  'C,  30  min 

16.  Implant,  Boron,  5  x  10 15,  32KeV 

17.  Plasma  Photoresist  Strip  (Oxygen) 
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18.  Photoresist  Stripper 

19.  Photoresist  Application 

20.  Prebake,  98  5C,  30  min 

21.  Mask  Level  IR 

22.  Photoresist  Development 

23.  Postbake,  120  °C,  30  min 

24.  Implant,  Phorsphorus,  5xlOn,  lOOKeV 

25.  Plasma  Photoresist  Strip  (Oxygen) 

26.  Photoresist  Stripper 

27.  BHF  Etch  10:1,  2  min 

•  Gate  Dielectric  Formation 

1.  RCA  Clean 

2.  Triple  Wall  Dry  Oxidation,  800  X  ,  900  °C 

3.  Photoresist  Application 

4.  Prebake,  98  °C,  30  min 

5.  Mask  Level  MW 

6.  Photoresist  Development 

7.  Postbake,  120  °C,  30  min 

8.  BHF  Etch,  10:1,  2  min 

9.  Photoresist  Stripper 

10.  RCA  Clean 

11.  Triple  Wall  Dry  Oxidation,  720  °C,  20  X  ,  9  min 

12.  LPCVD  Nitride,  250  mTorr,  735  °C,  120  X  ,  5  min,  10:1 

13.  Wet  Blocking  Oxidation,  1000  °C,  40  X  ,  50  min 

•  Gate  Material 

1.  LPCVD  Polysilicon,  800  mTorr,  180  seem  SiH4,  625  °C,  5000  X  ,  30  min 

2.  RCA  Clean 

3.  POCI3  Doping,  900  °C,  25  min  Pre-deposition,  30  min  Drive-in 

4.  BHF  Etch,  10:1 , 15  sec. 

5.  Photoresist  Application 

6.  Prebake,  98  3C,  30  min 

7.  Mask  Level  PY 

8.  Photoresist  Development 

9.  Postbake,  120  °C,  30  min 
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10.  Plasma  Polysilicon/Gate  Dielectric  Etch  (SF6) 

11.  BHF  Etch,  100:1,  1  min 

12.  Hot  H3P04  Etch,  170  °C,  3.5  min 

13.  BHF  Etch,  100:1,  1  min 

14.  Photoresist  Stripper 

•  Source/Drain  Formation 

1.  RCA  Clean 

2.  Dry  Pad  Oxidation,  900  °C,  200-300  X  ,  15  min 

3.  Photoresist  Application 

4.  Prebake,  98  °C,  30  min 

5.  Mask  Level  N+ 

6.  Photoresist  Development 

7.  Postbake,  120  °C,  30  min 

8.  Source/Drain  Implant,  Phorsphorus,  2  x  1015,  lOOKeV 

9.  Plasma  Photoresist  Strip  (Oxygen) 

10.  Photoresist  Stripper 

11.  Photoresist  Application 

12.  Prebake,  98  °C,  30  min 

13.  Mask  Level  P+ 

14.  Photoresist  Development 

15.  Postbake,  120  °C,  30  min 

16.  Source/Drain  Implant,  Boron,  5  x  1015,  30KeV 

17.  Plasma  Photoresist  Strip  (Oxygen) 

18.  Photoresist  Stripper 

19.  RCA  Clean  without  HF  Dip 

20.  Anneal  and  Drive-in,  950  °C,  60  min 

21.  BHF  Etch,  10:1,  1  min 

•  Contact  Window  Formation 

1.  RCA  Clean 

2.  Wet  Oxidation,  900  °C,  1000  \  ,  30  min 

3.  Photoresist  Application 

4.  Prebake,  98  °C,  30  min 

5.  Mask  Level  CW 

6.  Photoresist  Development 
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7.  Postbake,  120  °C,  30  min 

8.  BHF  Etch,  10:1,  3-5  min 

9.  Photoresist  Stripper 

10.  Dilute  HF  Etch  (HF  Dip),  30  sec 

•  Metallization 

1.  RF  Sputtering  Aluminum,  110  mTorr,  60  min 

2.  Photoresist  Application 

3.  Prebake,  98  °C,  30  min 

4.  Mask  Level  MET 

5.  Photoresist  Development 

6.  Postbake,  120  °C,  30  min 

7.  PAN  Etch,  45  °C,  2  min 

8.  Photoresist  Stripper 

9.  Backside  Clean-up 

10.  Backside  RF  Sputtering  Aluminum,  110  mTorr,  60  min 

11.  Preliminary  Check  ensuring  contact  window  open 

12.  Organic  Clean 

13.  PMA,  450  °C,  30  min 
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Appendix  C 


Media  Coverage 

From  Neural  Network  Today,  January,  1991 


EURAL 
ET  WORKS 


ODAY 


INC  COMPUTING  TECHNOLOGIES 


Jan  nary  1991 


DARPA  Funds  Academics,  Agencies,  National  Laboratories 


Academics  funded  by  the  Defense 
Advanced  Research  Project  Agency 
(DARPA)  gave  reviews  of  their  work 
at  the  project  update  meeting  Ian 
year,  along  with  industry 
contractees,  government  laborato¬ 
ries  and  agencies  such  as  the 
National  Aeronautics  and  Space 
Administration's  (NASA's)  Jet 

T  shnnemro  (TPT  > 


technologies,  as  well  as  theoretical 
advances. 

One  of  the  most  aggressive  thorn 
funded  by  DARPA  is  the  modifiable 
synapse.  Digital  menories  are  ill- 
suited  for  storing  the  analog 
memory  element  used  by  neurai 
networks  which  is  why  DARPA  is 


funding  the  futuristic  quest  for  an 
analog  memory  unit. 

Researcher  Marvin  White  at 
Lehigh  University  (Bethlehem, 
Penn.)  is  attempting  to  create  an 
electronic  version  of  the  synaose 
with  a  irnlP-diripr— rir  nili-rf  the  Socas 
Continued  an  page  4 


Neural  Networks  Call  Bets  at  Horse-Race  Track 


DARPA  Funds  Academics,  Agencies,  National  Labs  mabie  synapse  using  chips  with  a 

opacitor  t=n  scheme  and  11-bit 


Continued  from  page  1 
memory  transistor.  Sonos  is  an 
electa  caily-reprograminacle  nonvola¬ 
tile  method  of  adaptively  changmg  the 
conducance  of  an  analog  synapse. 
Compared  with  decrhcally-emsabie 
programmable  read-only  memories 
(EEPROMs),  it  offers  a  low  progmm- 
ming  voltage  (5  volts).  Sonos  also  has 
low  power  dissipation,  wide  dynamic 
range  and  strong  radiation  hardness. 
Sonos  can  mimic  biological  synapses 
with  reinforcement  learning  and  has 
stable  long-term  memory  retention. 

At  the  Massachusera  Institute  of 
Technology  (MIT)  investigator  Alice 
Otiang  is  using  chargecsupied  devices 
(CCDs)  as  the  analog  memory  element. 
MIT  is  developing  a  high-speed,  low- 
power,  geneni -purpose  feature- 
exu  actor  and  classifier  using  nemai 
technology.  MIT s  target  applications 
are  image-  and  speech-iecogniticn. 
Chiang  favors  CCDs  wtuch  tan  store 
analog  levels  of  information  in 
circulating  ’bucket  brigades'  with 
better  than  99999%  charge  tnnsfer 
efficiency  and  greater  than  45db 
dynamic  range. 

MIT  selected  a  generic  two-layer 
neural  network  classifier  based  on 


vector-matrix  products  :br  imeiemetn- 
lion.  It  can  also  be  used  Sot  l-D 
correlations  and  3-D  marched  Stem 
Two  versions  of  a  microchio  neuial 


processor  have  been  designed: 


•  6144  mncection  dassifier  with  '32 
incus  and  32  outputs 
The  ms:  veman  petibnss  2-0 
altering  a  ^ay-level  images  with  30 
programmable  7 -bv-7-by-8  bit  spatial 
Sites  which  can  exeac  features  bom 
an  input  image  In  real-time.  The  39 
squares  miiEmeter  chip  arm  consiss  of 
an  analog  input  buffer.  -19  multiplying 
D/A  convenes  and  20  7-by-7-by-8  bit 
local  memories.  The  10MHz  device 
runs  at  one  ration  arithmetic-open- 
tiens  per  second  at  less  than  I  warn 
A  second-!?  square  nuffimeter  cop 
assists  ca  analog  input  Iktife,  144 
Multipiing  a  Distal  Analog  Converses 
and  14  Wit  144-dement  memories. 

The  lOMHti  device  runs  at  3JbiIBen 
asthmetic  operations  per  second  and 
assumes  3  warts.  Future  vesiors  »ifl 
be  soled  us  usmg  similar  chips  In  a 
paraile  pipelined  configuration.  When 
mounted  on  boards,  the  duos  an  act 
as  high-speed  neural  coprocessors  for 
conventional  cgital  prorsaots. 

NASA's  Je  Fropuision  Laboratory 
(Pasadena.  Cdit)  is  investigating 
apactots  as  the  analog  memory 
element  in  neunl  netware.  Researcher 
.4nilThakocr  at  JPL  reported  on  a 
project  to  evahate  die  feasibility  of 
usmg  anaiog  hardware  to  impiemet 
neural  netwcnr  learning  methods. 

Then  areas  of  interest  cover  speech, 
pattern-  and  target-recognition,  sonar, 
procen  control  and  prediction. 

. — .  ■  ■ rS  -% 


resolution.  A  neuron  chip  performs  the 
sigmoid  and  provides  variable  gain.  The 
network  uses  36  neurons  febneareti 


with  CMCS  ensaam  VLSI  microchips 
connected  In  a  feedback  configuration. 
Setting  feedback  23  zero,  for  feed¬ 
forward  caiy,  yields  a  network  with 
eight  inpua.  right  outputs  and  32 
hidden  mots  in  up  to  8  hidden  layers. 


CunentiyJPL  is  experimenting  with 
teaming  that  sequentially  pmurfas  each 
synaptic  wegnt  under  a  computer 
control  The  onmputg  reads  the  enor 
in  the  output  ffnm  hardware  to 
generate  the  camroi  signal  for  the 
modifications  of  the  weights-  This 
architecture  has  been  successruDy  tested 
ca  the  invest  kinematics  transforma¬ 


tions  in  robotics  and  for  recognition  of 
at  .tain  features  hem  special  signatures. 

In  Lincoln  Labs  adjacent  to  MIT, 
researcher  Couresh  Mehanian  is 


designing  an  optical  neural  achnoicgy 
using  a  manoiithic  opto-riectronic 
transistor  (MCET).  A  neuron  consists  of 
a  multiple-quantum  well  (MQW)  light 
detector  on  the  inpus.  a  nonlinear 
tesonan  t-tunnriing  diode  and  another 
MQW  functioning  as  the  output 
modulator.  This  optical  neuron  sums 
iB  inputs,  preforms  a  sigmoidal 
aansfotrnanon  and  modulates  the 
output  optical  beam.  The  intensitycf 
the  beam  represents  neural  activity. 

Lincoln  Labs  propose  building 
microchip  arrays  at  sensing  MOETs 
which  can  be  ssruoreo  into  the  byes 
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DARPA  LOOKING  HARD  AT  NEURAL  NETS... 


(fs  gemng  more  and  mere 
difficult  oo  ignore  neural 
networks.  Just  ask  the 
Defense  Department,  whose 
Detense  Advanced  Research 
Projects  Agency  is  going  a 
spend  S33  million  through 
1992  to  see  if  the  netwoncs 
can  help  solve  signal-pro- 
oessing  problems. 

The  lure  of  neurai  nets, 
which  more  or  less  attack 
problems  the  wav  the 
human  brain  does,  is  thai 
they  do  not  need  cranpiese 
data  to  solve  aomplex  prob¬ 
lems — like  a  human,  they 
use  context  and  a  kind  of 
intuition.  And  that,  plus  mas¬ 
sive  parallelism  -aid  reaktme 


performance,  adds  up  to 
mere  accuracy  fer  missiles 
and  increased  maneuverabil¬ 
ity  for  tanks,  ships,  and  air¬ 
craft.  Whars  more,  there  is 
evidence  that  neural  nets 
degrade  mere  graceruflv  and 
are  easier  to  program  than 
conventional  ones. 

New,  Darpa  is  funding  a 
one-year  efbit  at  50  aaccpa- 
rnes.  bborarries.  and  unrver- 
sues.  They  are  working  on 
neural  simulation,  theory,  and 
modeling,  with  more  than 
haif  he  effcn  devoted  to  sim¬ 
ulating  automatic  target 
recognition  and  speech 
reccgnirioa  and  sonar  and 
sesnac  signal  kfcsai&arioaS 


HARDER  TIMES  IH 
MASSACHUSETTS 

The  widening  Massachu¬ 
setts  malaise  has  finally 
infected  Digital  Equipment 
Corp..  the  Maynard.  Mass., 
computer  giant  that  had 
never  had  involuntary  bvotfs 
in  its  33-vear  existence.  And 
analysts  believe  the  lay-off  of 
3.500  that  Digital  announced 
last  month  may  not  be 
enough  to  stem  a  slump  in 
earnings. 

Digital  wants  to  mm  its 
head  count  by  6.000  by  the 
end  of  its  fiscal  year.  June  50. 
A  voluntary  retirement  pro¬ 
gram  tell  some  3.500  shorn 
Digital  still  emplovs  more 
than  120.000  workhvide  and 
is  the  second  largest  empiev- 
er  in  Massachusetts,  behind 
Raytheon  Co.  But  Digital's 
downsizing  will  lurther  cut 
employment  in  the  Bav  State, 
which  has  lost  some  200.000 
lobs  m  the  last  year,  most  of 
them  in  the  once-soanng 
computer  and  electronics 
belt  along  the  Boston  areas 
stoned  Route  123.  Q 


.AND  HERE'S  NEURAL  COMPUTER  THAT  DOES  2.3  BILLION  OPERATiONS/S 


Even  as  me  Pentagon  s 
Defense  Acvanoed  Research 
Projects  Agency  seeds  the 
neural  network  pasures.  cor¬ 
porate  researchers  keep 
working  an  neurai  oompur- 
ers — machines,  modeled  on 
the  human  brain,  that  can 
handle  casks  requiring  intuit¬ 
ion — though  anything  like 
ccmmechl  anpiementanon 
is  years  jwav.  Now  Hinda 
Ltd.  says  a  has  come  up  with 
one  whew  learning  process¬ 
ing  unit  hat  cm  handle  2d 
biilion  operanens  s.  10  antes 
what  can  be  obtained  by 
simulating  a  neural  computer 
on  an  Hitacha  5-620  super¬ 
computer. 

The  Hitachi  lab  model 
includes  1.152  neurons  and 
is  ust  12  m.  high.  3.3  m. 
wide,  and  9  n.  deep.  The 
companv  has  developed 
stock-pnce  predetton  ana 
sigruture-venricition  appii- 
caoons  that  con  oe  run  on  a 
worksnocn  Ttar  s  linked  :o 
the  neurai  svstem.  A  stock- 
pnce  predkeen  rakes  10  s. 
savs  a  Hiochi  sookesman. 


and  a  signature  venficacon 
takes  2  s. 

The  machine  goes  a  long 
way  toward  overcoming 
faults  o i  existing  hardware- 
based  neural  computers: 
they  either  have  too  few  neu¬ 
rons.  or  they  leam  too  slow  ¬ 
ly.  A  practical  system  needs 


at  Irast  1,000  iniereonnecEd 
neurons,  say  researched  at 
Hindu 

The  new  computer  is 
based  on  an  LSI  circuit 
announced  in  1989  by 
Hitachi  that  houses  576  neu¬ 
rons.  Eight  of  them  are  used 
in  the  new  computer.  B 
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From  Brown  and  White,  April  28,  1992 


LU  scientists  seek  to  speed  up 
innovative  neural  net  processe 


-  By  JAN1NE  CONNOR  Elecaical  Engineering  Marvin  White 

So«ne*  Wnt»r  graduate  students  Chun- Yu  Malcolm 

The  lifestyle  of  the  Jetsons  is  not  that  Chen  and  Margaret  French, 
far  from  reality.  Innovations  familiar  to  Chen,  White  and  French  are  wort 
our  favorite  future  family,  such  as  to  develop  a  device  that  has  the  abilit 

computers  that  cm  recognize  handwrit-  "leant"  similar  to  the  way  people  do. 
ing,  verify  signatures,  translate  speech  Bono  do  this  means  imitating  the 
and  drive  cars  already  exist  thanks  to  brain's  IO-billion-oemon  oerwoit  ar 
artificial  neural  networks.  the  exzensive  neural  network  of 200, i 

Neural  networking  is  a  new  infonna-  nerves  throughout  the  body, 
rion  processing  technique  that  is  being  ‘Individual  neurons  from  our  bodi 
researched  to  a  great  extent  at  Lehigh.  It  are  relatively  slow  compared  to  comj 
__  borrows  its  basic  principles  from  .  ere,  but  their  network  architecture  m; 
biology,  but  is  itself  at  the  cutting  edge  their  processing  fast,"  Chen  said.  The 
of  computer  research.  Lehigh  researchers  are  trying  to  devc 

Among  those  involved  in  this  new  structures  to  increase  the  speed 

exciting  field  are  Sherman-Faiichitd 

professor  of  Computer  Science  and  See  NEURAL  NETS  Page  13 
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From  Page  1 1 

artificial  neural  network  structures. 

The  researchers  at  Sherman  Fairchild 
Lab  arc  developing  a  new  type  of  solid- 
state  electronic  neuron  with  funds  from 
the  Defense  Advanced  Research  Projects 
Agency  (DARPA). 

Lehigh.  Massachusetts  Institute  of 
Technology  and  Hebrew  University  are 
ihe  only  schools  that  ihe  DARPA 
Artificial  Intelligence  Neural  Networks 
Technology  Program  supports  finan¬ 


cially  in  this  specific  area  of  technology. 
Fellowship  support  from  the  NSF 
Engineering  Research  Cenier  for 
Advanced  Technology  for  Large 
Structural  Systems  (ATLSS)  is  also 
contributing  to  the  research. 

“This  research  will  be  an  ongoing 
project  for  many  years."  White  said. 
“Right  now  we  are  in  the  embryonic 
stages." 

"This  is  a  fairly  new  field  with  a  lot 
of  new  researchers,"  said  Chen,  who 
received  his  undergraduate  and  masters 
electrical  engineering  degrees  from 


Lehigh  and  is  now  working  on  neural 
networks  toward  his  Ph.D. 

“Wc  hope  what  wc  do  will  contribute 
to  state  of  the  art  research  regarding 
neural  network  technology,"  he  said. 

The  artificial  neural  networks 
researchers  are  designing  are  contained 
in  an  electronic  chip. 

By  applying  different  voltages,  the 
scientists  can  train  an  electrical  circuit  to 
produce  the  correct  output  for  changing 
input.  In  other  words,  the  circuit 
"leams"  how  to  respond  based  on  its 
prior  experiences. 


The  artificial  neural  networks 
developed  at  Sherman  Fairchild  Lab 
improve  upon  previous  work  hy  aipiir 
ing  a  low  programming  voltage  ol  5 
volts  and  cutting  down  on  loss  of 
current.  The  system's  small  size  and 
real-lime  application  possibilities  also 
make  it  attractive. 

Chen  said,  “l  ant  delighted  10  have 
the  opportunity  to  work  on  dm  project 
because  1  think  it  has  the  potential  to 
make  a  significant  impact  in  ihe  com¬ 
puter  industry  and  signal  processing 
field." 
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